Conversation
…ck in a deadlock on CI.
BenchmarksStartupParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 58 metrics, 7 unstable metrics. Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.59.0-SNAPSHOT~f6a54229fb, baseline=1.59.0-SNAPSHOT~189c0b6c76
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.084 s) : 0, 1083920
Total [baseline] (8.776 s) : 0, 8776349
Agent [candidate] (1.086 s) : 0, 1086337
Total [candidate] (8.759 s) : 0, 8759052
section iast
Agent [baseline] (1.235 s) : 0, 1235394
Total [baseline] (9.377 s) : 0, 9376582
Agent [candidate] (1.223 s) : 0, 1222505
Total [candidate] (9.335 s) : 0, 9334708
gantt
title insecure-bank - break down per module: candidate=1.59.0-SNAPSHOT~f6a54229fb, baseline=1.59.0-SNAPSHOT~189c0b6c76
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.185 ms) : 0, 1185
crashtracking [candidate] (1.182 ms) : 0, 1182
BytebuddyAgent [baseline] (651.344 ms) : 0, 651344
BytebuddyAgent [candidate] (651.279 ms) : 0, 651279
GlobalTracer [baseline] (282.643 ms) : 0, 282643
GlobalTracer [candidate] (284.712 ms) : 0, 284712
AppSec [baseline] (32.634 ms) : 0, 32634
AppSec [candidate] (32.751 ms) : 0, 32751
Debugger [baseline] (67.208 ms) : 0, 67208
Debugger [candidate] (67.462 ms) : 0, 67462
Remote Config [baseline] (635.387 µs) : 0, 635
Remote Config [candidate] (633.781 µs) : 0, 634
Telemetry [baseline] (8.958 ms) : 0, 8958
Telemetry [candidate] (9.008 ms) : 0, 9008
Flare Poller [baseline] (3.834 ms) : 0, 3834
Flare Poller [candidate] (3.847 ms) : 0, 3847
section iast
crashtracking [baseline] (1.2 ms) : 0, 1200
crashtracking [candidate] (1.181 ms) : 0, 1181
BytebuddyAgent [baseline] (799.653 ms) : 0, 799653
BytebuddyAgent [candidate] (791.372 ms) : 0, 791372
GlobalTracer [baseline] (258.528 ms) : 0, 258528
GlobalTracer [candidate] (256.146 ms) : 0, 256146
IAST [baseline] (27.306 ms) : 0, 27306
IAST [candidate] (26.797 ms) : 0, 26797
AppSec [baseline] (34.814 ms) : 0, 34814
AppSec [candidate] (33.867 ms) : 0, 33867
Debugger [baseline] (65.391 ms) : 0, 65391
Debugger [candidate] (65.13 ms) : 0, 65130
Remote Config [baseline] (570.784 µs) : 0, 571
Remote Config [candidate] (590.955 µs) : 0, 591
Telemetry [baseline] (8.475 ms) : 0, 8475
Telemetry [candidate] (8.477 ms) : 0, 8477
Flare Poller [baseline] (3.632 ms) : 0, 3632
Flare Poller [candidate] (3.57 ms) : 0, 3570
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.59.0-SNAPSHOT~f6a54229fb, baseline=1.59.0-SNAPSHOT~189c0b6c76
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.086 s) : 0, 1085625
Total [baseline] (10.783 s) : 0, 10782572
Agent [candidate] (1.09 s) : 0, 1089619
Total [candidate] (10.919 s) : 0, 10919090
section appsec
Agent [baseline] (1.265 s) : 0, 1264514
Total [baseline] (10.961 s) : 0, 10960764
Agent [candidate] (1.266 s) : 0, 1266129
Total [candidate] (10.995 s) : 0, 10995354
section iast
Agent [baseline] (1.227 s) : 0, 1226998
Total [baseline] (11.21 s) : 0, 11210290
Agent [candidate] (1.225 s) : 0, 1225397
Total [candidate] (11.227 s) : 0, 11226782
section profiling
Agent [baseline] (1.204 s) : 0, 1204038
Total [baseline] (10.883 s) : 0, 10883263
Agent [candidate] (1.204 s) : 0, 1204202
Total [candidate] (10.886 s) : 0, 10886167
gantt
title petclinic - break down per module: candidate=1.59.0-SNAPSHOT~f6a54229fb, baseline=1.59.0-SNAPSHOT~189c0b6c76
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.184 ms) : 0, 1184
crashtracking [candidate] (1.181 ms) : 0, 1181
BytebuddyAgent [baseline] (652.064 ms) : 0, 652064
BytebuddyAgent [candidate] (654.802 ms) : 0, 654802
GlobalTracer [baseline] (282.478 ms) : 0, 282478
GlobalTracer [candidate] (283.924 ms) : 0, 283924
AppSec [baseline] (32.676 ms) : 0, 32676
AppSec [candidate] (32.685 ms) : 0, 32685
Debugger [baseline] (68.228 ms) : 0, 68228
Debugger [candidate] (68.061 ms) : 0, 68061
Remote Config [baseline] (640.522 µs) : 0, 641
Remote Config [candidate] (628.35 µs) : 0, 628
Telemetry [baseline] (9.036 ms) : 0, 9036
Telemetry [candidate] (8.942 ms) : 0, 8942
Flare Poller [baseline] (3.786 ms) : 0, 3786
Flare Poller [candidate] (3.804 ms) : 0, 3804
section appsec
crashtracking [baseline] (1.187 ms) : 0, 1187
crashtracking [candidate] (1.183 ms) : 0, 1183
BytebuddyAgent [baseline] (691.658 ms) : 0, 691658
BytebuddyAgent [candidate] (691.144 ms) : 0, 691144
GlobalTracer [baseline] (258.293 ms) : 0, 258293
GlobalTracer [candidate] (258.732 ms) : 0, 258732
AppSec [baseline] (173.286 ms) : 0, 173286
AppSec [candidate] (174.208 ms) : 0, 174208
Debugger [baseline] (66.522 ms) : 0, 66522
Debugger [candidate] (66.874 ms) : 0, 66874
Remote Config [baseline] (729.831 µs) : 0, 730
Remote Config [candidate] (775.482 µs) : 0, 775
Telemetry [baseline] (9.286 ms) : 0, 9286
Telemetry [candidate] (9.341 ms) : 0, 9341
Flare Poller [baseline] (3.69 ms) : 0, 3690
Flare Poller [candidate] (3.801 ms) : 0, 3801
IAST [baseline] (24.47 ms) : 0, 24470
IAST [candidate] (24.651 ms) : 0, 24651
section iast
crashtracking [baseline] (1.21 ms) : 0, 1210
crashtracking [candidate] (1.186 ms) : 0, 1186
BytebuddyAgent [baseline] (793.158 ms) : 0, 793158
BytebuddyAgent [candidate] (792.267 ms) : 0, 792267
GlobalTracer [baseline] (256.877 ms) : 0, 256877
GlobalTracer [candidate] (256.805 ms) : 0, 256805
AppSec [baseline] (35.255 ms) : 0, 35255
AppSec [candidate] (34.539 ms) : 0, 34539
Debugger [baseline] (65.35 ms) : 0, 65350
Debugger [candidate] (65.489 ms) : 0, 65489
Remote Config [baseline] (594.197 µs) : 0, 594
Remote Config [candidate] (579.91 µs) : 0, 580
Telemetry [baseline] (8.536 ms) : 0, 8536
Telemetry [candidate] (8.457 ms) : 0, 8457
Flare Poller [baseline] (3.569 ms) : 0, 3569
Flare Poller [candidate] (3.531 ms) : 0, 3531
IAST [baseline] (27.056 ms) : 0, 27056
IAST [candidate] (27.141 ms) : 0, 27141
section profiling
ProfilingAgent [baseline] (96.486 ms) : 0, 96486
ProfilingAgent [candidate] (97.234 ms) : 0, 97234
crashtracking [baseline] (1.211 ms) : 0, 1211
crashtracking [candidate] (1.235 ms) : 0, 1235
BytebuddyAgent [baseline] (701.777 ms) : 0, 701777
BytebuddyAgent [candidate] (701.844 ms) : 0, 701844
GlobalTracer [baseline] (220.978 ms) : 0, 220978
GlobalTracer [candidate] (220.676 ms) : 0, 220676
AppSec [baseline] (32.106 ms) : 0, 32106
AppSec [candidate] (32.016 ms) : 0, 32016
Debugger [baseline] (68.338 ms) : 0, 68338
Debugger [candidate] (68.189 ms) : 0, 68189
Remote Config [baseline] (664.721 µs) : 0, 665
Remote Config [candidate] (609.833 µs) : 0, 610
Telemetry [baseline] (8.961 ms) : 0, 8961
Telemetry [candidate] (8.836 ms) : 0, 8836
Flare Poller [baseline] (3.768 ms) : 0, 3768
Flare Poller [candidate] (3.711 ms) : 0, 3711
Profiling [baseline] (97.068 ms) : 0, 97068
Profiling [candidate] (97.818 ms) : 0, 97818
LoadParameters
See matching parameters
SummaryFound 3 performance improvements and 1 performance regressions! Performance is the same for 16 metrics, 16 unstable metrics.
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.59.0-SNAPSHOT~f6a54229fb, baseline=1.59.0-SNAPSHOT~189c0b6c76
dateFormat X
axisFormat %s
section baseline
no_agent (1.182 ms) : 1171, 1194
. : milestone, 1182,
iast (3.237 ms) : 3194, 3281
. : milestone, 3237,
iast_FULL (5.865 ms) : 5806, 5923
. : milestone, 5865,
iast_GLOBAL (3.702 ms) : 3647, 3757
. : milestone, 3702,
profiling (1.988 ms) : 1969, 2008
. : milestone, 1988,
tracing (1.796 ms) : 1781, 1811
. : milestone, 1796,
section candidate
no_agent (1.187 ms) : 1176, 1199
. : milestone, 1187,
iast (3.133 ms) : 3096, 3171
. : milestone, 3133,
iast_FULL (5.927 ms) : 5867, 5986
. : milestone, 5927,
iast_GLOBAL (3.56 ms) : 3508, 3612
. : milestone, 3560,
profiling (1.951 ms) : 1935, 1968
. : milestone, 1951,
tracing (1.77 ms) : 1755, 1785
. : milestone, 1770,
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.59.0-SNAPSHOT~f6a54229fb, baseline=1.59.0-SNAPSHOT~189c0b6c76
dateFormat X
axisFormat %s
section baseline
no_agent (17.867 ms) : 17682, 18051
. : milestone, 17867,
appsec (19.796 ms) : 19594, 19997
. : milestone, 19796,
code_origins (17.746 ms) : 17571, 17921
. : milestone, 17746,
iast (17.843 ms) : 17664, 18021
. : milestone, 17843,
profiling (19.822 ms) : 19624, 20020
. : milestone, 19822,
tracing (17.615 ms) : 17437, 17792
. : milestone, 17615,
section candidate
no_agent (19.7 ms) : 19495, 19906
. : milestone, 19700,
appsec (18.829 ms) : 18635, 19023
. : milestone, 18829,
code_origins (17.736 ms) : 17557, 17914
. : milestone, 17736,
iast (17.794 ms) : 17616, 17973
. : milestone, 17794,
profiling (18.933 ms) : 18744, 19122
. : milestone, 18933,
tracing (17.675 ms) : 17498, 17852
. : milestone, 17675,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.59.0-SNAPSHOT~f6a54229fb, baseline=1.59.0-SNAPSHOT~189c0b6c76
dateFormat X
axisFormat %s
section baseline
no_agent (15.01 s) : 15010000, 15010000
. : milestone, 15010000,
appsec (15.036 s) : 15036000, 15036000
. : milestone, 15036000,
iast (17.962 s) : 17962000, 17962000
. : milestone, 17962000,
iast_GLOBAL (17.868 s) : 17868000, 17868000
. : milestone, 17868000,
profiling (15.185 s) : 15185000, 15185000
. : milestone, 15185000,
tracing (14.837 s) : 14837000, 14837000
. : milestone, 14837000,
section candidate
no_agent (15.554 s) : 15554000, 15554000
. : milestone, 15554000,
appsec (14.634 s) : 14634000, 14634000
. : milestone, 14634000,
iast (18.111 s) : 18111000, 18111000
. : milestone, 18111000,
iast_GLOBAL (17.705 s) : 17705000, 17705000
. : milestone, 17705000,
profiling (14.791 s) : 14791000, 14791000
. : milestone, 14791000,
tracing (14.668 s) : 14668000, 14668000
. : milestone, 14668000,
Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.59.0-SNAPSHOT~f6a54229fb, baseline=1.59.0-SNAPSHOT~189c0b6c76
dateFormat X
axisFormat %s
section baseline
no_agent (1.472 ms) : 1460, 1483
. : milestone, 1472,
appsec (3.63 ms) : 3415, 3846
. : milestone, 3630,
iast (2.217 ms) : 2152, 2283
. : milestone, 2217,
iast_GLOBAL (2.259 ms) : 2193, 2324
. : milestone, 2259,
profiling (2.067 ms) : 2013, 2121
. : milestone, 2067,
tracing (2.051 ms) : 2000, 2103
. : milestone, 2051,
section candidate
no_agent (1.467 ms) : 1456, 1479
. : milestone, 1467,
appsec (3.689 ms) : 3470, 3908
. : milestone, 3689,
iast (2.209 ms) : 2144, 2275
. : milestone, 2209,
iast_GLOBAL (2.254 ms) : 2188, 2320
. : milestone, 2254,
profiling (2.07 ms) : 2017, 2123
. : milestone, 2070,
tracing (2.044 ms) : 1992, 2096
. : milestone, 2044,
|
bric3
left a comment
There was a problem hiding this comment.
THis is ok for me, but I'll wait on IAST folk to approve there
|
|
||
| // Check logger class by name to avoid NoClassDefFoundError at runtime for tests without | ||
| // Logback. | ||
| if (LOGGER.getClass().getName().equals("ch.qos.logback.classic.Logger")) { |
There was a problem hiding this comment.
question: Mostly to IAST folk, does it makes sense to support additional loggers ?
There was a problem hiding this comment.
Since Taintable.DebugLogger is only used internally in tests (activated via TaintableVisitor.DEBUG = true) and not exposed to end users, we only need it to work with whatever logger is present in our test suite
mcculls
left a comment
There was a problem hiding this comment.
LGTM, just needs a review from IAST before merging
What Does This Do
Use
slf4j-simpleinstead oflogbackthat has a high chance of getting stuck in a deadlock on CI.We are using very old version of
logback = "1.2.13"that is only version compatible with Java 8 and has a high chance to hang on CI on modern JVMs (usually 11).Motivation
Green CI.
Additional Notes
With probability of 10% task is hanging on CI:
With stack trace showing that it is hanging inside
logback: